Application of WHAM to Black Sea Bass

.title[
# Application of WHAM to Black Sea Bass
]
.subtitle[
## Black sea bass Research Track
]
.author[
### Tim Miller, Kierten Curti, Alex Hansell
]

---

.footnote[U.S. Department of Commerce | National Oceanic and Atmospheric Administration | National Marine Fisheries Service]

code.cpp{
  font-size: 14px;
}
code.r{
  font-size: 14px;
}

</style>

</style>

---
#Outline <br>

* Description of proposed base model
* Diagnostics
* Results
* Path to the base model
  * Initial bridge building
  * Progression of model development

---

# Description of proposed base model

---

.footnote[U.S. Department of Commerce | National Oceanic and Atmospheric Administration | National Marine Fisheries Service]

---

# General attributes

* 2 regions
  * North
  * South
* 2 stock components:
  * North
  * South
* Model years: 1989 - 2021
* Ages: 1-8+
* 2 Environmental processes (1959-2022)
  * Bottom temperature in North
  * Bottom temperature in South
* Natural mortality = 0.4 all ages, components, regions

---

#Movement configuration:

<img src="data:image/png;base64,#plots/migration_diagram_1.jpg" width="80%" style="display: block; margin: auto;" />
* All Jan 1 recruitment for a given stock component only in respective regions
* North fish can only move from south to north in first 5 seasons
* North fish can only move from north to south in last 4 seasons
* Any remaining North fish that are in the south move back to their spawning region at end of season 5
* All North fish remain in North spawning region until end of season 7
* Spawning season is only time when whole North population is in spawning region
* South population stays in South.

---

#Movement configuration:

* The monthly/seasonal movement matrix after spawning:
`$$\mathbf{p}_{1} = \begin{bmatrix} 1-p_1 & p_1 \\ 0 & 1 \\ \end{bmatrix}$$`
* Before spawning:
`$$\mathbf{p}_{2} = \begin{bmatrix} 1 &  0 \\ p_2 & 1-p_2 \\ \end{bmatrix}$$`

---

#Movement rates from Stock Synthesis

* The Stock Synthesis model has 2 seasons (6 months each) 
* proportion `$P_1$` of the northern component moves to the south in one season and some proportion `$P_2$` move back to the south in the second season. 
* The movement matrices for each season are `$$\mathbf{P}_{1} = \begin{bmatrix} 1-P_1 & P_1 \\ 0 & 1 \\ \end{bmatrix}$$`
and `$$\mathbf{P}_{2} = \begin{bmatrix} 1 &  0 \\ P_2 & 1-P_2 \\ \end{bmatrix}$$`

---

#Transforming to shorter seasons in WHAM

* Approximate the monthly movement matrices as the roots of `$\mathbf{P}_1$` and `$\mathbf{P}_2$` defined by the number of months of movement for each season (5 and 4, respectively): 
  * Given the proportion parameter, the eigen decomposition of the matrices can be used to define the roots `$$\mathbf{P}_1^{\frac{1}{5}} = \mathbf{V}_1 \mathbf{D}_1^{\frac{1}{5}} \mathbf{V}_1^{-1}$$` `$$\mathbf{P}_2^{\frac{1}{4}} = \mathbf{V}_2 \mathbf{D}_2^{\frac{1}{4}} \mathbf{V}_2^{-1}$$` where `$\mathbf{V}_i$` and `$\mathbf{D}_i$` are the matrix of eigenvectors (columnwise) and the diagonal matrix of corresponding eigenvalues of `$\mathbf{P}_i$` for parameter `$P_i$`.

---

#Parameterizing the prior distribution

* The actual SS parameter estimates `$x_1=-1.44$` and `$x_2=1.94$` are transformations of `$P_1=0.11$` and `$P_2=0.78$` such that
`$$P_i = \frac{1}{1 + 2e^{-x_i}}$$`
* Multi-WHAM uses an additive logit transformation (simply a logit transformation when there are only two regions): 
`$$p_i = \frac{1}{1+e^{-y_i}}$$`
---

#Parameterizing the movement prior distributions

Used a parametric bootstrap approach:
* Simulate 1000 values from a normal distribution with mean and standard deviation defined by the SS parameter estimate and standard error `$\tilde x_i \sim N(x_i, SE(x_i))$`. 
* For each simulated value 
  * construct `$\mathbf{P}_i$`, 
  * take the appropriate root,
  * calculated inverse logit for `$\tilde y_i$`. 
  * calculate the mean and SD of the simulated values `$\tilde y_i$`. 
  * mean values did not differ meaningfully from the transformation of the original estimates: `$y_1 = -3.79$` and `$y_2 = -0.79$`
  * SD was approximately 0.2 for both parameters.
* distributions for random effects defining the movement parameters configured using the mean and SD from the bootstraps.
---

#Initial abundance at age

* With the movement configuration, northern origin fish (ages 2+) can occur in the southern region on January 1. 
* Estimating initial numbers at age as separate parameters can be challenging even in single-stock models. 
* To avoid difficulties, we used the equilibrium assumption described previously. 
* Two parameters are estimated for each regional stock component: an initial recruitment and an equilibrium full F across all fleets.

---

#Recruitment and Survival/movement transitions

* 2DAR1 (age and year) correlated random effects for both the northern and southern components.
* Variance and correlation parameters are different for the northern and southern components.
* Northern component:
  * abundance at age 1 on January 1 (recruitment) is only allowed in the northern region, 
  * older individuals may occur in either region on Jan 1 (based on movement description)
  * survival random effects will occur for abundances at age in both regions. 
  * Base model assumes very small variance for the transitions in the southern region (approximately SCAA)
    * 2DAR1 models with estimated variance for this region would not converge (correlation could not be estimated).

---

# Observations

* Aggregate catch: 2 fleets in each region:
  * Commercial (1989-2021)
  * Recreational (1989-2021)
* Aggregate indices: 2 in each region:
  * Spring VAST (1989-2021)
  * Recreational CPA (1989-2021)
* Age composition for all fleets and indices (1989-2021)
* Model-based Bottom temperature observation in each region (1959-2022)

---

# Selectivity

`$$\\[12pt]$$`

<table>
 <thead>
  <tr>
   <th style="text-align:left;"> Data component </th>
   <th style="text-align:left;"> Mean Selectivity model </th>
   <th style="text-align:left;"> Random effects configuration </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> North Commercial </td>
   <td style="text-align:left;"> age-specific (flat-topped at ages > 3) </td>
   <td style="text-align:left;"> 2D-AR1 (age and year) </td>
  </tr>
  <tr>
   <td style="text-align:left;"> North Recreational </td>
   <td style="text-align:left;"> age-specific (flat-topped at ages > 6) </td>
   <td style="text-align:left;"> 2D-AR1 (age and year) </td>
  </tr>
  <tr>
   <td style="text-align:left;"> South Commercial </td>
   <td style="text-align:left;"> logistic </td>
   <td style="text-align:left;"> None </td>
  </tr>
  <tr>
   <td style="text-align:left;"> South Recreational </td>
   <td style="text-align:left;"> logistic </td>
   <td style="text-align:left;"> None </td>
  </tr>
  <tr>
   <td style="text-align:left;"> North Recreational CPA </td>
   <td style="text-align:left;"> age-specific (flat-topped at ages > 1) </td>
   <td style="text-align:left;"> AR1 (year) </td>
  </tr>
  <tr>
   <td style="text-align:left;"> North VAST </td>
   <td style="text-align:left;"> age-specific (flat-topped at ages > 4) </td>
   <td style="text-align:left;"> 2D-AR1 (age and year) </td>
  </tr>
  <tr>
   <td style="text-align:left;"> South Recreational CPA </td>
   <td style="text-align:left;"> age-specific (flat-topped at ages > 2) </td>
   <td style="text-align:left;"> None </td>
  </tr>
  <tr>
   <td style="text-align:left;"> South VAST </td>
   <td style="text-align:left;"> age-specific (flat-topped at ages > 1) </td>
   <td style="text-align:left;"> None </td>
  </tr>
</tbody>
</table>

---

# Age composition models

`$$\\[12pt]$$`

<table>
 <thead>
  <tr>
   <th style="text-align:left;"> Data component </th>
   <th style="text-align:left;"> Age Composition Likelihood </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> North Commercial </td>
   <td style="text-align:left;"> Dirichlet-Multinomial </td>
  </tr>
  <tr>
   <td style="text-align:left;"> North Recreational </td>
   <td style="text-align:left;"> Logistic-normal (0s as missing) </td>
  </tr>
  <tr>
   <td style="text-align:left;"> South Commercial </td>
   <td style="text-align:left;"> Logistic-normal (AR1, 0s as missing) </td>
  </tr>
  <tr>
   <td style="text-align:left;"> South Recreational </td>
   <td style="text-align:left;"> Logistic-normal (AR1, 0s as missing) </td>
  </tr>
  <tr>
   <td style="text-align:left;"> North Recreational CPA </td>
   <td style="text-align:left;"> Logistic-normal (0s as missing) </td>
  </tr>
  <tr>
   <td style="text-align:left;"> North VAST </td>
   <td style="text-align:left;"> Dirichlet-Multinomial </td>
  </tr>
  <tr>
   <td style="text-align:left;"> South Recreational CPA </td>
   <td style="text-align:left;"> Logistic-normal (AR1, 0s as missing) </td>
  </tr>
  <tr>
   <td style="text-align:left;"> South VAST </td>
   <td style="text-align:left;"> Logistic-normal (AR1, 0s as missing) </td>
  </tr>
</tbody>
</table>
---

#Uncertainty in Rec CPA indices

* CVs provided by analyses that generated Rec CPA indices were deemed implausibly small (CVs: 0.02 to 0.06).
* In many early runs and proposed base model we estimated a scalar of the SD of the log-aggregate Rec CPA indices.
  * Estimates of the scalar were usually approximately 5 for the north and the south Rec CPA indices.
* Estimation included in the proposed base model allow more realistic estimates of uncertainty in model output.

---

#Bottom Temperature effects on recruitment

* Included model-based bottom temperature observations in the BSB model.
* Very small uncertainty in observations (SEs: 0.03 to 0.09).
* State-space treatment: 
  * Modeled latent covariate as AR1 process.
`$$X_y \sim N\left(\mu_X(1-\rho_X) + \rho_X X_{y-1}, \sigma_X^2\right)$$`
  * Observations of the latent covariate:
`$$x_y \sim N\left(X_{y}, \sigma_x^2\right)$$`
* Fit models with and without effects on northern and southern recruitment `$$\log R_y = \mu_R + \beta X_y + \epsilon_y.$$`
  * Examined AIC for full model and all retrospective peels.
* Might be worth exploring an alternative effect on `$M$` for age 1.
  * Would also affect reference points and projections differently.

---

# Diagnostics for base model

---

.footnote[U.S. Department of Commerce | National Oceanic and Atmospheric Administration | National Marine Fisheries Service]

---

# Jitter analysis

* Simulated 100 fixed effects parameter vectors from a normal distribution with mean equal to the optimized values and covariance equal to the hessian-based covariance matrix of the optimzed model. 
* All of the re-fits of the mdoel resulted in the same marginal negative log-likelihood. 
* The gradients at these optimized values were all satisfactory with maximum absolute values less the `$10^{-9}$`.

---

#Jitter analysis

---

#Self-test

* Simulated new observations conditional on all random effects estimated in the proposed base model 
* Fitted the same model configuration each of the simulated data sets. 
* For 7 of the the simulated data sets the model failed to optimize. 
* The maximum absolute gradient was `$<10^{-6}$` for only 9 and `$<10^{-4}$` for 52 of the 93 successfully fitted models. 
* The poor convergence appeared to be attributable to the estimation of the scalar for the standard errors of the log-transformed northern Recreational CPA index
  * Estimates tended to 0 for nearly all of the fits ($<0.01$ for 83 fits). 
* Even across all fits including those with poor convergence, the SSB estimates appeared to be reliable.

---
#Self-test

<img src="data:image/png;base64,#bsbrt_peer_review_bsbapp_files/figure-html/unnamed-chunk-7-1.png" style="display: block; margin: auto;" />
---

#MASE

* We fit 7 configurations of the model where the last 1 to 7 years of aggregate index and age composition observations were removed sequentially (peels). 
* Calculate the mean absolute scaled error (MASE) of the predictions at 1 to 5 years beyond the final year of index observations (horizons). 
  * The mean absolute errors are scaled by the mean absolute errors of so-called naïve predictions using the aggregate index observation from the final year of each peel. 
* A MASE < 1 results when the mean absolute error is greater using the model than the naïve forecast. 
* For the proposed base model, predictions for 3 of the 4 surveys performed similarly to naïve predictions across all horizons, 
* BUT MASE scores were much greater than 1 for the northern region recreational CPA index at all horizons. 
  * The large MASE values occur because the index has no trend and low variability over the years used for the calculation whereas the model predictions vary much more.

---

#MASE

<img src="data:image/png;base64,#../2023.RT.Runs/Run34/mase_WHAM_for_unnamed_stock.png" width="50%" style="display: block; margin: auto;" />
---

#MASE
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/predicted_vs_obs_WHAM_for_unnamed_stock.png" width="50%" style="display: block; margin: auto;" />
---

#One-step-ahead residuals

* For composition observations and for aggregate observations in state-space models, Pearson residuals do not have the appropriate properties: 
  * independent standard normal for a correctly specified model
* Due to lack of independence of observations.
* One-step-ahead (OSA) residuals have this property
* However, understanding causes of mis-specification can be difficult.

---

#OSA: North commercial fleet

.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/OSA_resid_catch_4panel_North_Commercial.png" width="100%" style="display: block; margin: auto;" />
]
---

#OSA: North commercial fleet

* some indication of trends in residuals of some of the age composition observations early in the time series and for the the first age class.

.pull-left[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/OSA_resid_paa_6panel_North_Commercial.png" width="90%" style="display: block; margin: auto;" />
]
.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/Catch_age_comp_osa_resids_North_Commercial.png" width="90%" style="display: block; margin: auto;" />
]
---

#OSA: North recreational fleet

.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/OSA_resid_catch_4panel_North_Recreational.png" width="100%" style="display: block; margin: auto;" />
]
---

#OSA: North recreational fleet

* residuals showed some tendency of underdispersion

.pull-left[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/OSA_resid_paa_6panel_North_Recreational.png" width="90%" style="display: block; margin: auto;" />
]
.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/Catch_age_comp_osa_resids_North_Recreational.png" width="90%" style="display: block; margin: auto;" />
]
---

#OSA: North Rec CPA index

.pull-left[
* no signs of mis-specification
]
.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/OSA_resid_catch_4panel_North_REC_CPA.png" width="100%" style="display: block; margin: auto;" />
]
---

#OSA: North Rec CPA index

* no signs of mis-specification

.pull-left[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/OSA_resid_paa_6panel_North_REC_CPA.png" width="90%" style="display: block; margin: auto;" />
]
.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/Catch_age_comp_osa_resids_North_REC_CPA.png" width="90%" style="display: block; margin: auto;" />
]
---

#OSA: North VAST index

.pull-left[
* some evidence of tendency toward negative residuals
]
.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/OSA_resid_catch_4panel_North_VAST_Spring.png" width="100%" style="display: block; margin: auto;" />
]
---

#OSA: North VAST index

* some indication of positive residuals, particularly at age 1

.pull-left[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/OSA_resid_paa_6panel_North_VAST_Spring.png" width="90%" style="display: block; margin: auto;" />
]
.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/Catch_age_comp_osa_resids_North_VAST_Spring.png" width="90%" style="display: block; margin: auto;" />
]
---

#OSA: South commercial fleet

.pull-left[
* residuals tended to be negative
]
.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/OSA_resid_catch_4panel_South_Commercial.png" width="100%" style="display: block; margin: auto;" />
]
---

#OSA: South commercial fleet

* no evidence of mis-specification

.pull-left[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/OSA_resid_paa_6panel_South_Commercial.png" width="90%" style="display: block; margin: auto;" />
]
.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/Catch_age_comp_osa_resids_South_Commercial.png" width="90%" style="display: block; margin: auto;" />
]
---

#OSA: South recreational fleet

.pull-left[
* residuals were under-dispersed but no trends. 
]
.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/OSA_resid_catch_4panel_South_Recreational.png" width="100%" style="display: block; margin: auto;" />
]
---

#OSA: South recreational fleet

* somewhat underdispersed, but no trends

.pull-left[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/OSA_resid_paa_6panel_South_Recreational.png" width="90%" style="display: block; margin: auto;" />
]
.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/Catch_age_comp_osa_resids_South_Recreational.png" width="90%" style="display: block; margin: auto;" />
]
---

#OSA: South Rec CPA index

.pull-left[
* somewhat under-dispersed
]
.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/OSA_resid_catch_4panel_South_REC_CPA.png" width="100%" style="display: block; margin: auto;" />
]
---

#OSA: South Rec CPA index

* a few large negative residuals at older ages, but otherwise no apparent trends

.pull-left[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/OSA_resid_paa_6panel_South_REC_CPA.png" width="90%" style="display: block; margin: auto;" />
]
.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/Catch_age_comp_osa_resids_South_REC_CPA.png" width="90%" style="display: block; margin: auto;" />
]
---

#OSA: South VAST index

.pull-left[
* somewhat over-dispersed
]
.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/OSA_resid_catch_4panel_South_VAST_Spring.png" width="100%" style="display: block; margin: auto;" />
]
---

#OSA: South VAST index

* some trend with observed proportion and age

.pull-left[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/OSA_resid_paa_6panel_South_VAST_Spring.png" width="90%" style="display: block; margin: auto;" />
]
.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/diagnostics/Catch_age_comp_osa_resids_South_VAST_Spring.png" width="90%" style="display: block; margin: auto;" />
]
---

#Retrospective patterns

* Seven peels of the base model.
* Strong retrospective patterns in the most recent management track assessment for the northern component of the stock, do not occur in base model.

---

#Retrospective patterns: North SSB
.pull-left[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/retro/BSB_North_SSB_retro.png" width="100%" style="display: block; margin: auto;" />
]
.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/retro/BSB_North_SSB_retro_relative.png" width="100%" style="display: block; margin: auto;" />
]
---

#Retrospective patterns: North F (average of ages 6-7)
.pull-left[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/retro/North_Fbar_retro.png" width="100%" style="display: block; margin: auto;" />
]
.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/retro/North_Fbar_retro_relative.png" width="100%" style="display: block; margin: auto;" />
]
---

#Retrospective patterns: South SSB
.pull-left[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/retro/BSB_South_SSB_retro.png" width="100%" style="display: block; margin: auto;" />
]
.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/retro/BSB_South_SSB_retro_relative.png" width="100%" style="display: block; margin: auto;" />
]
---

#Retrospective patterns: South F (average of ages 6-7)
.pull-left[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/retro/South_Fbar_retro.png" width="100%" style="display: block; margin: auto;" />
]
.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/retro/South_Fbar_retro_relative.png" width="100%" style="display: block; margin: auto;" />
]
---

#Results

---

.footnote[U.S. Department of Commerce | National Oceanic and Atmospheric Administration | National Marine Fisheries Service]

---

# SSB, R: North

<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/results/SSB_Rec_time_BSB_North.png" width="50%" style="display: block; margin: auto;" />
---

# SSB, R: South

<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/results/SSB_Rec_time_BSB_South.png" width="50%" style="display: block; margin: auto;" />
---

# SSB, R: Total

<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/results/SSB_Rec_time_total.png" width="50%" style="display: block; margin: auto;" />
---

# SSB, F

<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/results/SSB_F_trend.png" width="50%" style="display: block; margin: auto;" />
---

#F by fleet

<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/results/F_byfleet.png" width="50%" style="display: block; margin: auto;" />
---

# Bottom temperature

.pull-left[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/results/Ecov_1_North_BT.png" width="100%" style="display: block; margin: auto;" />
]
.pull-right[
<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/results/Ecov_2_South_BT.png" width="100%" style="display: block; margin: auto;" />
]
---

# Northern recruitment and bottom temperature

<img src="data:image/png;base64,#bsbrt_peer_review_bsbapp_files/figure-html/unnamed-chunk-26-1.png" width="50%" style="display: block; margin: auto;" />
---

# NAA random effects deviations

<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/results/NAA_dev_tile.png" width="100%" style="display: block; margin: auto;" />
---

#Selectivity

<img src="data:image/png;base64,#../2023.RT.Runs/Run34/plots_png/results/SelAA_tile.png" width="50%" style="display: block; margin: auto;" />
---

# Prior and posterior movement rates

![](data:image/png;base64,#bsbrt_peer_review_bsbapp_files/figure-html/move-prior-posterior-1.png)

---

# Path to the base model

---

.footnote[U.S. Department of Commerce | National Oceanic and Atmospheric Administration | National Marine Fisheries Service]

---

#Comparing ASAP with standard WHAM and Multi-WHAM

.pull-left[
* Based on data used in last management track 
  * Time series of observations not updated
  * Use previous fleet definitions (Trawl and Non-Trawl).
* Compared
  * Management track ASAP 
  * Separate fits in standard WHAM
  * Simultaneous (separate) fits in Multi-WHAM
]

.pull-right[
<img src="data:image/png;base64,#plots/compare_one_stock_models.png" width="100%" style="display: block; margin: auto;" />
]
---

# Bridge runs

Separate fits for north and south regions
* 1: Turn off all indices but NEFSC Spring BTS and Rec CPA
* 2: Update fishery catches, fishing fleets and catch WAA estimates to comm/rec fleets
* 3: Update Spring BTS and Rec CPA
* 4: Add 2020-2021
* 5: Update maturity
* 6: Add NEAMAP
* 7: Update remaining spring state indices (added VAST as well but didn’t turn them on)
* <span style='color: #B3B3B3;'>8: Rec CPA and both spring and fall VAST</span>
* 9: Rec CPA and VAST spring only (also a combined stock run that matches the single stock results. This combined run will be used for later runs.)

---

# Bridge runs

Separate fits for north and south regions
* <span style='color: red;'>1: Turn off all indices but NEFSC Spring BTS and Rec CPA</span> 
* <span style='color: red;'>2: Update fishery catches, fishing fleets and catch WAA estimates to comm/rec fleets</span>
* <span style='color: red;'>3: Update Spring BTS and Rec CPA</span> 
* 4: Add 2020-2021
* 5: Update maturity
* 6: Add NEAMAP
* 7: Update remaining spring state indices (added VAST as well but didn’t turn them on)
* <span style='color: #B3B3B3;'>8: Rec CPA and both spring and fall VAST</span>
* 9: Rec CPA and VAST spring only (also a combined stock run that matches the single stock results. This combined run will be used for later runs.)

---

# Bridge runs 1, 2, and 3

.pull-left[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs1-3.north.F.png" alt="North" width="100%" />
<p class="caption">North</p>
</div>
]
.pull-right[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs1-3.south.F.png" alt="South" width="100%" />
<p class="caption">South</p>
</div>
]
---

# Bridge runs 1, 2, and 3

.pull-left[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs1-3.north.SSB.png" alt="North" width="100%" />
<p class="caption">North</p>
</div>
]
.pull-right[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs1-3.south.SSB.png" alt="South" width="100%" />
<p class="caption">South</p>
</div>
]
---

# Bridge runs 1, 2, and 3

.pull-left[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs1-3.north.Rect.png" alt="North" width="100%" />
<p class="caption">North</p>
</div>
]
.pull-right[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs1-3.south.Rect.png" alt="South" width="100%" />
<p class="caption">South</p>
</div>
]
---

# Bridge runs

Separate fits for north and south regions
* 1: Turn off all indices but NEFSC Spring BTS and Rec CPA
* 2: Update fishery catches, fishing fleets and catch WAA estimates to comm/rec fleets
* <span style='color: red;'>3: Update Spring BTS and Rec CPA</span>
* <span style='color: red;'>4: Add 2020-2021</span>
* 5: Update maturity
* 6: Add NEAMAP
* 7: Update remaining spring state indices (added VAST as well but didn’t turn them on)
* 8: Rec CPA and both spring and fall VAST
* 9: Rec CPA and VAST spring only (also a combined stock run that matches the single stock results. This combined run will be used for later runs.)

---

# Bridge runs 3 and 4

.pull-left[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs3-4.north.F.png" alt="North" width="100%" />
<p class="caption">North</p>
</div>
]
.pull-right[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs3-4.south.F.png" alt="South" width="100%" />
<p class="caption">South</p>
</div>
]
---

# Bridge runs 3 and 4

.pull-left[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs3-4.north.SSB.png" alt="North" width="100%" />
<p class="caption">North</p>
</div>
]
.pull-right[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs3-4.south.SSB.png" alt="South" width="100%" />
<p class="caption">South</p>
</div>
]
---

# Bridge runs 3 and 4

.pull-left[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs3-4.north.Rect.png" alt="North" width="100%" />
<p class="caption">North</p>
</div>
]
.pull-right[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs3-4.south.Rect.png" alt="South" width="100%" />
<p class="caption">South</p>
</div>
]
---

# Bridge runs

Separate fits for north and south regions
* 1: Turn off all indices but NEFSC Spring BTS and Rec CPA
* 2: Update fishery catches, fishing fleets and catch WAA estimates to comm/rec fleets
* 3: Update Spring BTS and Rec CPA
* <span style='color: red;'>4: Add 2020-2021</span>
* <span style='color: red;'>5: Update maturity</span>
* <span style='color: red;'>6: Add NEAMAP</span>
* <span style='color: red;'>7: Update remaining spring state indices (added VAST as well but didn’t turn them on)</span>
* 8: Rec CPA and both spring and fall VAST
* 9: Rec CPA and VAST spring only (also a combined stock run that matches the single stock results. This combined run will be used for later runs.)

---

# Bridge runs 4, 5, 6, and 7

.pull-left[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs4-7.north.F.png" alt="North" width="100%" />
<p class="caption">North</p>
</div>
]
.pull-right[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs4-7.south.F.png" alt="South" width="100%" />
<p class="caption">South</p>
</div>
]
---

# Bridge runs 4, 5, 6, and 7

.pull-left[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs4-7.north.SSB.png" alt="North" width="100%" />
<p class="caption">North</p>
</div>
]
.pull-right[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs4-7.south.SSB.png" alt="South" width="100%" />
<p class="caption">South</p>
</div>
]
---

# Bridge runs 4, 5, 6, and 7

.pull-left[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs4-7.north.Rect.png" alt="North" width="100%" />
<p class="caption">North</p>
</div>
]
.pull-right[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs4-7.south.Rect.png" alt="South" width="100%" />
<p class="caption">South</p>
</div>
]
---

# Bridge runs

Separate fits for north and south regions
* 1: Turn off all indices but NEFSC Spring BTS and Rec CPA
* 2: Update fishery catches, fishing fleets and catch WAA estimates to comm/rec fleets
* 3: Update Spring BTS and Rec CPA
* 4: Add 2020-2021
* 5: Update maturity
* 6: Add NEAMAP
* <span style='color: red;'>7: Update remaining spring state indices (added VAST as well but didn’t turn them on)</span>
* <span style='color: red;'>8: Rec CPA and both spring and fall VAST</span>
* <span style='color: red;'>9: Rec CPA and VAST spring only (also a combined stock run that matches the single stock results. This combined run will be used for later runs.)</span>

---

# Bridge runs 7, 8, and 9

.pull-left[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs7-9.north.F.png" alt="North" width="100%" />
<p class="caption">North</p>
</div>
]
.pull-right[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs7-9.south.F.png" alt="South" width="100%" />
<p class="caption">South</p>
</div>
]
---

# Bridge runs 7, 8, and 9

.pull-left[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs7-9.north.SSB.png" alt="North" width="100%" />
<p class="caption">North</p>
</div>
]
.pull-right[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs7-9.south.SSB.png" alt="South" width="100%" />
<p class="caption">South</p>
</div>
]
---

# Bridge runs 7, 8, and 9

.pull-left[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs7-9.north.Rect.png" alt="North" width="100%" />
<p class="caption">North</p>
</div>
]
.pull-right[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#../Bridge.Runs/Comparison_Figures/Runs7-9.south.Rect.png" alt="South" width="100%" />
<p class="caption">South</p>
</div>
]
---

# Early research track runs (1-18)

* See table in the working paper
* Used updated and new observations 
* Included surveys as either separate indices or combined in fall and spring VAST indices (other than the recreational CPA)
* Explored alternative selectivity and age comp likelihood assumptions: reduce patterns in age comp OSA residuals. 
* WG determined to use aggregate VAST indices that account for changes in catchability.
* **However, age comp for fall VAST and NEAMAP were incorrectly calculated**
* **Runs 19+ only used the spring VAST and Rec CPA indices.**

---

# Middle runs (19-27)

* Issues with patterns in OSA residuals remained 
* Sometimes large retrospective patterns for the northern component. 
* After Run 27, we used separate models for north and south to more efficiently explore alternative assumptions about selectivity and age composition likelihoods
* **These analyses resulted in assumptions that remained the same across all later runs**
  * **Different age composition model assumptions for some of the fleets and indices.**
  * **Use of selectivity random effects for the northern fleets and indices.**

---

# Late runs (28-34)

* **Corrected some previous assumptions about movement.**
  * Rates from SS were not corrected for differences in seasonal time steps
  * Rates of movement for northern component to and from north were both occurring over all seasonal intervals outside of spawning
  * Incorrect movement rate from south to north.
* Runs 30+ assume negligible variance of survival random effects for northern origin fish occurring in the south on Jan 1.
  * Allows yearly AR1 correlation to be estimated.
* Runs that did not converge well:
  * Run 31 assumed temporal random effects on the movement rate from the north to the south
  * Run 32 assumed a Beverton-Holt stock recruit relationships for both the north and south components
* Run 33 investigated bottom temperature effects on recruitment and we found evidence of an effect on northern recruitment.  
* Run 34 relaxed estimated scalar for uncertainty in the Recreational CPA indices
* **After these runs we discovered a coding error in constructing initial numbers at age under an equilibrium assumption. 
  * Small changes in results Run 34: absolute differences in annual SSB estimates were less than 7%. 
  * **Refit Runs 30, 33, and 34 with the corrected model** 
    * None of the choices among these models would have changed.

---

#Evidence for bottom temperature effects on recruitment

`$$\\[12pt]$$`

.pull-left[
<table>
 <thead>
  <tr>
   <th style="text-align:left;">   </th>
   <th style="text-align:right;"> AIC </th>
   <th style="text-align:right;"> North `$\widehat{\sigma}_R$` </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> No effect </td>
   <td style="text-align:right;"> -1551.79 </td>
   <td style="text-align:right;"> 0.92 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> North temperature effect only </td>
   <td style="text-align:right;"> -1562.14 </td>
   <td style="text-align:right;"> 0.74 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Both temperature effects </td>
   <td style="text-align:right;"> -1561.61 </td>
   <td style="text-align:right;"> 0.74 </td>
  </tr>
</tbody>
</table>
]
.pull-right[
<table>
 <thead>
  <tr>
   <th style="text-align:right;"> Peel </th>
   <th style="text-align:right;"> No effect </th>
   <th style="text-align:right;"> North only </th>
   <th style="text-align:right;"> North and South </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:right;"> 0 </td>
   <td style="text-align:right;"> 10.35 </td>
   <td style="text-align:right;"> 0.00 </td>
   <td style="text-align:right;"> 0.53 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 10.17 </td>
   <td style="text-align:right;"> 0.00 </td>
   <td style="text-align:right;"> 0.38 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 2 </td>
   <td style="text-align:right;"> 9.22 </td>
   <td style="text-align:right;"> 0.00 </td>
   <td style="text-align:right;"> 0.65 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 3 </td>
   <td style="text-align:right;"> 9.29 </td>
   <td style="text-align:right;"> 0.00 </td>
   <td style="text-align:right;"> 0.56 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 4 </td>
   <td style="text-align:right;"> 9.07 </td>
   <td style="text-align:right;"> 0.00 </td>
   <td style="text-align:right;"> 0.70 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 5 </td>
   <td style="text-align:right;"> 8.53 </td>
   <td style="text-align:right;"> 0.00 </td>
   <td style="text-align:right;"> 0.31 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 6 </td>
   <td style="text-align:right;"> 8.20 </td>
   <td style="text-align:right;"> 0.36 </td>
   <td style="text-align:right;"> 0.00 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 7 </td>
   <td style="text-align:right;"> 7.78 </td>
   <td style="text-align:right;"> 0.11 </td>
   <td style="text-align:right;"> 0.00 </td>
  </tr>
</tbody>
</table>
]

---

# Comparison of Runs 30, 33 and 34
* Run 30: like run 34, but fixed Rec CPA CVs, no temperature effects on northern recruitment.
* Run 33: like Run 34, but fixed Rec CPA CVs
* The estimates of SSB, fully-selected fishing mortality, and recruitment were very similar for Runs 30, 33, and 34

---

# Comparison of Runs 30, 33 and 34: SSB

<img src="data:image/png;base64,#bsbrt_peer_review_bsbapp_files/figure-html/SSB-compare-1.png" style="display: block; margin: auto;" />
---

# Comparison of Runs 30, 33 and 34: Fishing mortality

<img src="data:image/png;base64,#bsbrt_peer_review_bsbapp_files/figure-html/F-compare-1.png" style="display: block; margin: auto;" />
---

# Comparison of Runs 30, 33 and 34: Recruitment

---

#Expected recruitment with and without temperature effects

<img src="data:image/png;base64,#bsbrt_peer_review_bsbapp_files/figure-html/unnamed-chunk-55-1.png" width="50%" style="display: block; margin: auto;" />
---

# Comparison of Runs 33 and 34

* Allowing the scalar for the standard error of the log-transformed Rec CPA indices to be estimated in the proposed base model results in a slight increase in uncertainty of spawning stock biomass and recruitment estimates.
* fitted models with `$(\text{SE}_1)$` and without `$(\text{SE}_2)$` the scalar of the Recreational CPA index standard errors estimated.
<img src="data:image/png;base64,#bsbrt_peer_review_bsbapp_files/figure-html/relative-se-ssb-R-1.png" style="display: block; margin: auto;" />